Search engines have emerged as heavily used tools that allow users thereof to quickly locate information that is pertinent to respective information retrieval goals of the users. Accordingly, a search engine is expected by users to handle a broad range of different types of information retrieval requests. Ranking algorithms (rankers) utilized in conventional search engines, however, are trained as general-purpose (one-size-fits-all) rankers with a single ranking function being applied to all queries. Inevitably, then, a search engine can provide search results responsive to receipt of a query that are not particularly relevant to the information retrieval goal of a user who issued the query to the search engine.
With more specificity, a ranker typically considers various features of a query and documents when ranking documents responsive to receipt of the query. When ranking the documents, the ranker assigns learned respective weights to the aforementioned features, wherein the weights are learned during a training procedure to optimize performance of the ranker with respect to some known information retrieval metric, such as normalized discounted cumulative gain (NDCG). The weights are learned through utilization of labeled training data, wherein the labeled training data includes labeled query/document relevance judgments. Weights of features considered by the ranker when ranking search results are then learned such that the overall performance of the ranker, with respect to labeled validation data, is optimized for the information retrieval metric.
As discussed above, since the ranker is learned to optimize information retrieval for all types of queries in the training data, there may exist certain types of queries with respect to which the ranker, and thus the search engine, performs relatively poorly. Revenue generated by a search engine is tied to user traffic; accordingly, it is desirable to attract new users to the search engine as well as to retain existing users. Therefore, it can be ascertained that improving performance of search engines with respect to various different types of queries is desirable.